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Abstract 

Background: Numerous studies on publication bias in clinical drug research have been undertaken, particularly on 
the association between sponsorship and favourable outcomes. However, no standardized methodology for the 
classification of outcomes and sponsorship has been described. Dissimilarities and ambiguities in this assessment 
impede the ability to compare and summarize results of studies on publication bias. To guide authors undertaking 
such studies, this paper provides recommendations for a uniform assessment of publication bias related to funding 
source. 

Methods and results: As part of ongoing research into publication bias, 472 manuscripts on randomised 
controlled trials (RCTs) with drugs, submitted to eight medical journals from January 2010 through April 2012, were 
reviewed. Information on trial results and sponsorship was extracted from manuscripts. During the start of this 
evaluation, several problems related to the classification of outcomes, inclusion of post-hoc analyses and follow-up 
studies of RCTs in the study sample, and assessment of the role of the funding source were encountered. A 
comprehensive list of recommendations addressing these problems was composed. To assess internal validity, 
reliability and usability of these recommendations were tested through evaluation of manuscripts submitted to 
journals included in our study. 

Conclusions: The proposed recommendations represent a first step towards a uniform method of classifying trial 
outcomes and sponsorship. This is essential to draw valid conclusions on the role of the funding source in 
publication bias and will ensure consistency across future studies. 

Keywords: Publication bias, Clinical drug research, Sponsorship, Pharmaceutical industry, Trial outcomes, 
Recommendations, Uniform assessment 



Background 

Introduction 

In recent years, the issue of publication bias in clinical 
drug research has been widely addressed. Studies re- 
porting results that are positive or favourable to the ex- 
perimental drug are more likely to be published, and 
outcomes that are statistically significant have higher 
odds of being fully reported [1-3]. Funding by the 
pharmaceutical industry and financial conflicts of inte- 
rest among authors have particularly been associated 
with publication of favourable outcomes [4-6]. Publication 
bias could result from researchers and sponsors failing to 
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submit trials with negative results, or from favouritism 
towards publication of positive results among peer re- 
viewers, journal editors and publishers. 

Numerous studies on the association between spon- 
sorship and trial outcomes have been undertaken [6]. 
Classification of outcomes as positive or negative and 
establishing the role of the funding source may appear 
to be straightforward. However, when reviewing studies 
on publication bias, no standardized methodology for 
the assessment of outcomes and sponsorship has been 
described. Definitions used for classification of trial out- 
comes are inconsistent across studies and the terms 
'sponsored by industry' and 'conflict of interest' have 
been applied in different ways. 

In a majority of the studies on publication bias, methods 
for classification of trial outcomes are only reported to a 
very limited extent, though there are positive exceptions 
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[5,7]. However, this assessment is not without hurdles, for 
instance when reviewing results of non-inferiority or 
equivalence trials and trials reporting safety rather than ef- 
ficacy outcomes. Some studies concentrate on results 
reported for primary endpoints, while others examine 
conclusion sections to classify outcomes. Some investiga- 
tors briefly mention that included trials are categorized 
according to funding source [7-9], while in other papers 
explicit criteria are given for the classification based on 
sponsor type. Drug trials are often considered to be 
industry-sponsored when funded by a pharmaceutical 
company [6,10-12], while some researchers also regard 
donation of study medication by a manufacturer as 
industry-sponsorship [6,10,11,13]. In addition, drug 
trials are sometimes classified as industry-sponsored if 
authored by one or more employees of the company 
manufacturing the test drug [10,11,13,14]. Further- 
more, the terms industry-funded, industry-supported, 
and industry-sponsored are used interchangeably when 
categorizing trials according to sponsorship. 

Dissimilarities and ambiguities in the classification of 
outcomes and sponsorship impede the ability to com- 
pare and summarize results of studies on publication 
bias. A standardized approach, taking into account dif- 
ferent types of financial and material support and a clear 
definition of trial outcomes, is essential to draw valid 
conclusions on the role of the funding source in publica- 
tion bias. To guide authors undertaking studies on the 
association between sponsorship and study outcomes or 
authors undertaking systematic reviews or meta-analyses 
of such studies, this paper provides recommendations 
for a uniform assessment of publication bias related to 
funding source. 

Methods 

As part of ongoing research into biased reporting of trial 
outcomes, we assessed the role of editorial and peer 
review processes in publication bias. We evaluated whe- 
ther submitted manuscripts with negative outcomes 
were less likely to be published than studies with positive 
outcomes, and determined the influence of sponsorship 
in this context. We retrospectively reviewed manuscripts 
submitted to eight journals from January 2010 through 
April 2012. One general medical journal (British Medical 
Journal) and seven specialty journals (Annals of the 
Rheumatic Diseases, British Journal of Ophthalmology, 
Diabetologia, Gut, Heart, Journal of Hepatology, and 
Thorax) were included. Journals were selected based on 
impact factor (journals indexed with the highest impact 
factors within subject categories, according to Institute 
for Scientific Information Journal Citation Report 2011), 
and the number of drug RCTs published in 2010-2011. 
Manuscripts reporting results of RCTs were included, if 
at least one study arm assessed the efficacy or safety of a 



drug intervention and a statistical test was used to evalu- 
ate treatment effects (n = 472). Two authors (MvL, HJO) 
extracted information on trial results and financial or 
material support from manuscripts. 

During the start of this evaluation, we encountered 
several problems related to (1) the classification of out- 
comes, (2) inclusion of post-hoc and subgroup analyses 
and follow-up studies of RCTs in the study sample, and 
(3) assessment of the role of the funding source. These 
problems hampered unequivocal judgment on sponsor- 
ship and the direction of trial outcomes. We composed 
a list of recommendations addressing these problems, to 
facilitate a standardized assessment of manuscripts in- 
cluded in our study and future studies on publication bias. 
To assess internal validity, reliability and usability of the 
recommendations were tested through evaluation of ma- 
nuscripts submitted to journals included in our study. 
Minor adjustments were made during this process, and a 
comprehensive list of recommendations was created. 

Findings 

Identified problems 
Classification of trial outcomes 

Poorly described methodology and lack of (or multiple) 
primary endpoints A poor description of trial design in 
research papers complicates the selection of manuscripts 
that are eligible for inclusion when conducting a study 
on publication bias. As trial outcomes are often clas- 
sified based on results for reported primary endpoints, 
failure to accurately define a primary endpoint among a 
plurality of outcomes described or mentioning multiple 
primary endpoints constitutes a problem. Guidelines for 
accurate reporting of RCTs are available by means of the 
CONSORT Statement (Consolidated Standards of Re- 
porting Trials) [15], but the quality of reporting regarding 
these and other aspects remains sub-optimal after publi- 
cation of CONSORT recommendations [16,17]. 

Safety as primary endpoint An issue not related to the 
adequacy of reporting, but to the type of outcomes 
reported, is represented by trials with safety rather than ef- 
ficacy as a primary endpoint. It could be argued whether 
trial results should be classified as positive when the test 
drug is proven to be as safe as the control treatment, or 
whether this would only be valid when less adverse events 
are reported for the test drug than for the comparator. 
Furthermore, it can be difficult to determine whether 
reported outcomes should be considered as efficacy or 
safety endpoints. 

Non-inferiority and equivalence trials Problems re- 
lated to trial design also apply to outcomes of non- 
inferiority and equivalence trials. In case non-inferiority 
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or equivalence is established for treatments in these 
trials, outcomes could be considered positive, as they 
confirm the hypothesis of the trial. However, there are 
no clinically relevant differences between treatments 
with regard to the primary outcome of the trial, which 
could also be interpreted as a negative finding. 

Inclusion of post-hoc analyses and follow-up studies of trials 

It could be argued whether post-hoc and subgroup ana- 
lyses and follow-up studies of RCTs should be included 
in studies on publication bias, next to original trial re- 
ports. Due to inadequate reporting on methodological 
aspects as described above, it can be difficult to discri- 
minate between original reports and secondary analyses 
of trial data based on publications in the scientific litera- 
ture. Unless original trial protocols are available, it is 
often not possible with a high degree of certainty to 
make judgments on what is post-hoc and what is not. 
However, post-hoc and subgroup analyses and follow-up 
studies of RCTs could be equally prone to influences of 
sponsorship and direction of results on publication of 
studies. 

Role of the funding source 

Different levels of support Assessment of the role of 
the funding source in drug trials is complex because of 
the variety of levels of involvement reported, ranging 
from financial support and donation of study medication 
by a pharmaceutical company, to input of the manufac- 
turer in the trial design, conduct, data analysis and pub- 
lication of results. Financial support from industry is 
often described as unrestricted educational grants, which 
suggests that funding was given without any predefined 
involvement of the hinder. Furthermore, some trials are 
authored by one or more employees of the company 
manufacturing the test drug or funding the study. 

Inadequate reporting of sponsorship The assessment 
of sponsorship can be complicated by the fact that some 
journals do not report funding sources or do not explain 
the sponsor's role in a trial, and authors may not dis- 
close received funding. Consequently, trials receiving 
industry funding could be falsely classified as non- 
industry. The CONSORT 2010 Statement and the Uni- 
form Requirements for Manuscripts Submitted to Bio- 
medical Journals of the International Committee of 
Medical Journal Editors (ICMJE) [15,18], both adopted 
by many journals, insist on reporting of information on 
funding sources and other support and the role of 
funders. However, not all authors adhere to these guide- 
lines when submitting manuscripts, and even if authors 
do adhere to them, journal editors may not require the 
material to be reported. 



Formal sponsorship It is often unclear based on the in- 
formation reported in manuscripts whether trials should 
be regarded as investigator-initiated studies or that phar- 
maceutical companies qualify for formal sponsorship as 
defined in the ICH-GCP (International Conference on 
Harmonisation of Technical Requirements for Registra- 
tion of Pharmaceuticals for Human Use - Good Clinical 
Practice) guidelines [19]. According to these guidelines, 
a study sponsor is "an individual, company, institution, 
or organization which takes responsibility for the initi- 
ation, management, and/or financing of a clinical trial." 
Therefore, financial support only does not necessarily 
imply formal sponsorship by the donor. In the public 
trial registries ClinicalTrials.gov, EU Clinical Trials 
Register and ISRCTN Register, definitions for funding 
and sponsorship are not consistent across registries and 
do not unequivocally discriminate between trials for- 
mally sponsored by industry according to ICH-GCP 
guidelines, and those receiving only financial or material 
support. 

Non-profit trials When absence of industry support is 
evident, it can subsequently be difficult to distinguish 
between trials sponsored by governmental sources ver- 
sus other nonprofit organisations. Frequently, it is un- 
clear which party carries responsibility regarding quality 
and safety measures during trials. Furthermore, trials 
can be supported by non-industry organisations, that are 
owned by pharmaceutical companies. 

Recommendations 
Classification of trial outcomes 

Outcomes of drug RCTs should be classified as positive 
if results for the primary endpoint (as reported) are sta- 
tistically significant (p < 0.05 or 95% confidence interval 
[CI] for difference excluding 0 or 95% CI for ratio ex- 
cluding 1) and supports the efficacy of the test drug; or 
negative if the result does not reach statistical signifi- 
cance or is statistically significant in the direction of the 
control treatment being more efficacious (Table 1). 
When a trial is designed as an equivalence or non infer- 
iority study and the two treatments are equivalent, i.e. 
differences in treatment effects do not equal or exceed 
preset margins, results should be classified as positive. 
When treatments are equally effective regarding the pri- 
mary endpoint and it is not clearly stated whether a trial 
was designed as superiority or non-inferiority trial, 
results should be classified as positive. For manuscripts 
describing a safety parameter as primary endpoint, trials 
in which the test drug is reported to be as safe or safer 
than the control treatment (either placebo or active 
comparator) should be considered as positive. When 
authors explicitly hypothesized that the test drug is 
expected to be less harmful than control, only trials 
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Table 1 Classification of outcomes of drug RCTs based on 
results reported for primary endpoints 

Positive Negative 
outcome outcome 



X 
X 



X 



X 



reporting results that support the safety of the test drug 
should be classified as positive. If treatments are equally 
harmful, results should be classified as negative. 

If no primary outcome is stated for a trial, or multiple 
primary endpoints are selected, results should be clas- 
sified based on the statistical significance and direction 
of most (primary) outcomes (>50%). When two co- 
primary outcomes are stated, results for both endpoints 
should support the efficacy or safety of the test drug in 
order to classify a trial as positive. If results for only one 
of the two endpoints are statistically significant in favour 
of the test drug, results should be classified as negative. 
When treatment effects are not directly compared bet- 
ween groups in the statistical analysis (i.e. against base- 
line measurements in each treatment arm), results 
should be classified as positive if they are statistically sig- 
nificant in favour of the test drug. If uncertainty remains 
over the classification of trial outcomes, studies should 
be scored as negative, unless the overall tone of the con- 
clusion section favours the test drug. 



Inclusion of post-hoc analyses and follow-up studies of trials 

Studies explicitly referred to as post-hoc or subgroup 
analyses and follow-up studies of single RCTs are eligible 
for inclusion in studies on publication bias, if they com- 
pare the efficacy or safety of treatments to which parti- 
cipants have been assigned in an RCT, but with regard 
to other (secondary) outcomes, limited to a specific part 
of the original study population, or with an extended 
follow-up in these analyses. Post-hoc and subgroup ana- 
lyses based on combined data of multiple RCTs should 
not be included. 

Role of the funding source 

Drug trials should be considered as non-industry studies 
when no associations with a pharmaceutical company 
can be identified in the trial manuscript. No author- 
industry affiliations are reported, no manufacturer is 
named in the materials and methods section or acknow- 
ledgements for providing study medication or placebos 
free of charge, and no statements are made about finan- 
cial support from pharmaceutical companies (Table 2). 
These RCTs are sponsored by governmental or other 
non-profit organizations, including universities, hospitals 
and foundations. 

Trials should be classified as industry-sponsored when a 
pharmaceutical company is explicitly described as study 
sponsor in the manuscript, or when the company funding 
the trial is reported to have participated in the study de- 
sign, data collection, analysis, preparation of the manu- 
script, and/or decision to publish. Involvement of authors 
employed by a pharmaceutical company is indicative of 
industry sponsorship, but not sufficient to qualify a trial as 
industry-sponsored. Studies reporting donation of study 
medication or receipt of financial support from a pharma- 
ceutical company, and trials with one or more authors 
employed by the company manufacturing the test drug 
should be assigned to a separate class of industry-sup- 
ported trials. Sole provision of grants or drugs does not 
fully comply with criteria for sponsorship according to 
ICH-GCP guidelines, and thus differs from formal spon- 
sorship by a pharmaceutical company. However, when a 
trial is funded by industry and there are company- 
employed co-authors, the company will likely participate 
in the conduct and reporting of the trial. These trials 
should be classified as industry-sponsored, if the role of 
the funding source is not explicitly reported. Trials funded 
or otherwise supported by a nonprofit organisation owned 
by a pharmaceutical company should be categorized as in- 
dustry-supported. Investigator-initiated post-hoc analyses 
of single RCTs that were formally sponsored by industry 
should also be scored as industry-supported. When doubt 
remains over sponsorship and the trial is registered, infor- 
mation in the trial registry should take precedence over 
other sources of information. 



Results for primary endpoint statistically X 

significant and supporting the efficacy of test 

drug 

Results for primary endpoint do not reach 
statistical significance 

Results for primary endpoint statistically 
significant in direction of control treatment 
being more efficacious 

Treatments equivalent regarding primary X 
endpoint in non-inferiority or equivalence trials 

Treatments equally effective regarding primary X 
endpoint in trials not explicitly described as 
superiority or non-inferiority study 

Test drug as safe or safer than control treatment X 

in trials with safety parameter as primary 

endpoint 

Treatments equally harmful in trials with safety 
parameter as primary endpoint, when 
hypothesized that test drug is expected to be 
safer than control 

Results for >50% of (primary) endpoints X 
statistically significant in favour of test drug, 
when no/multiple primary endpoints are 
reported 

Results for one primary endpoint statistically 
significant in favour of test drug, when two 
co-primary endpoints are reported 

Treatment effects not compared between X 
groups but against baseline in each arm; results 
for primary endpoint statistically significant in 
favour of test drug 
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Table 2 Classification of drug RCTs according to sponsor type 



Non-industry trial 



Industry-supported 
trial 



Industry-sponsored 
trial 



Pharmaceutical company explicitly reported as study sponsor in 
manuscript 

Company funding the trial participated in design, conduct, 
analysis, writing of article and/or decision to publish 

Trial funded by industry and author(s) affiliated to industry, but 
role of funding source not reported 

Financial support received from pharmaceutical company 

Donation of study medication or placebos by manufacturer 

One or more authors employed by pharmaceutical company 
manufacturing the test drug 



7. Trial supported by nonprofit organisation owned by 
pharmaceutical company 

8. Investigator-initiated post-hoc analysis of single RCT formally 
sponsored by industry 

9. Pharmaceutical industry not in any way involved in trial 
Formal study sponsor according to ICH-GCP guidelines 

Responsible party for quality and safety measures during trial 



Nonprofit 
organisation 

Nonprofit 
organisation 



X 
X 

X 
X 
X 



Nonprofit organisation Pharmaceutical company 
Nonprofit organisation Pharmaceutical company 



Discussion 

In this study, several problems related to the classification 
of outcomes and sponsorship were identified. We have 
formulated recommendations addressing these problems 
to facilitate a uniform assessment of manuscripts included 
in studies on publication bias. 

In this article, we have focused on the assessment of 
publication bias related to funding source. However, the 
relationship between favourable outcomes and spon- 
sorship is much more complex. Numerous systematic re- 
views have found that industry sponsorship is associated 
with results that are favourable to the product of the com- 
pany funding the trial [4,20,21]. Besides publication bias, 
there are other ways by which outcomes may be 
influenced. Industry bias may occur through the choice of 
comparators, dosing and timing of comparisons, coding of 
events and selective data analysis, interpretation of data 
and selective outcome reporting [6,21,22]. Although we 
have concentrated on publication bias, our recommenda- 
tions are generally applicable to studies comparing indus- 
try versus non-industry trials in relation to favourable 
outcomes. Furthermore, we only considered drug RCTs, 
but the recommendations are equally relevant to device 
studies, as these are also often sponsored by companies 
with a financial interest in the study outcomes. 

In our recommendations, reported results are classi- 
fied as positive or negative. Trials in which primary end- 
points are reached or study hypotheses are confirmed 
are considered to be positive. Possible alternatives to 
categorize trials would include 'significant' vs 'non-sig- 
nificant] or 'favorable' vs 'unfavorable' findings. However, 



use of these terms would impede the classification of 
non-inferiority or equivalence trials and studies with 
safety parameters as the primary endpoint. We have not 
included a third 'neutral' category, as we aimed to keep 
the classification as straightforward as possible. More- 
over, previous studies on publication bias found rela- 
tively low numbers of studies with neutral or unclear 
results [23-25], so we included these in the 'negative' 
category. 

For the classification of outcomes, we have concen- 
trated on results reported for the primary endpoint. 
However, in several studies on industry bias, trials are 
categorized as positive if the conclusion is favourable 
(i.e. the authors recommend the test drug), not the 
reported results [13]. In this article, we recommend con- 
sidering conclusions only when trials cannot be classified 
based on reported results. Previous studies reporting on 
concordance between study results and conclusions 
found that industry-sponsored trials were less concor- 
dant than non- industry studies [6]. This lack of concor- 
dance, and the finding that industry-sponsored studies 
are more likely to have favourable conclusions [6], may 
be explained by the use of distorted presentation or 
'spin' in papers [26]. As results may be more objective 
measures of treatment effects, we chose to concentrate 
on the assessment of results. Future studies on publi- 
cation bias should clearly state whether trial results or 
conclusions are being examined. 

Trials were classified as non-industry, industry-sup- 
ported or industry- sponsored. One of the most fun- 
damental differences between industry-supported and 
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industry-sponsored studies relates to the overall responsi- 
bility for the conduct of the trial as defined by ICH-GCP 
guidelines. Industry-supported trials are done under the 
responsibility of non-profit organizations in contrast to 
industry-sponsored studies. This distinction is usually not 
made in previous literature but is very relevant in terms of 
quality assessment of reported research. We have not 
included a category of 'sponsorship not stated'. Many 
journals nowadays ask for information on received fun- 
ding and the role of the funding source, and require that 
manuscripts on RCTs conform to ICMJE requirements 
and CONSORT guidelines. Therefore, most trials recently 
submitted should report on funding and be registered in a 
trial registry, making information on sponsorship trace- 
able. However, the non-industry category may include tri- 
als that received industry support, if authors failed to 
disclose funding. In a recent Cochrane review on sponsor- 
ship and research outcomes, studies were also coded as 
non-industry if it was not reported who sponsored the 
study. The review authors stated that some of these trials 
were likely industry-sponsored, but no changes in results 
were seen when studies without sponsorship statements 
were excluded [6]. In addition, industry-sponsored studies 
may be misclassified as industry-supported, as apparently 
independently conducted industry-supported trials may 
have unreported sponsor involvement [27] . 

For future studies on publication bias, it is important 
that the quality of reporting on design and sponsorship 
of trials is improved. Journals should actively enforce the 
CONSORT Statement and ICMJE requirements, and 
could require authors to complete a form for disclosure 
of sponsorship and support received for the reported 
trial [28]. 

We emphasize that our recommendations are only vali- 
dated for use by three investigators. In order to prove 
whether the recommendations are acceptable to re- 
searchers in general, it is essential that the external validity 
of this proposed classification system is determined in a 
study undertaken by different researchers on a different 
sample of manuscripts. To establish external validity, a 
Delphi consensus technique could be conducted among a 
larger group of researchers. Future research should focus 
on whether the classification system is adopted in practice 
by investigators undertaking studies on trial outcomes and 
sponsorship. 

In conclusion, the recommendations proposed in this 
article represent a first step towards a uniform method of 
classifying trial outcomes and sponsorship. This is essen- 
tial to draw valid conclusions on the role of the funding 
source in publication bias and will ensure consistency 
across future studies. 
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